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Abstract — Convolutional low-density parity-check (LDPC) 
codes (or spatially-coupled codes) have now been shown 
to achieve capacity on binary-input memoryless symmetric 
channels. The principle behind this surprising result is the 
threshold-saturation phenomenon, which is defined by the belief- 
propagation threshold of the spatially-coupled ensemble saturat- 
ing to a fundamental threshold defined by the uncoupled system. 

Previously, the authors demonstrated that potential functions 
can be used to provide a simple proof of threshold saturation for 
coupled scalar recursions. In this paper, we present a simple proof 
of threshold saturation that applies to a wide class of coupled 
vector recursions. The conditions of the theorem are verified for 
the density-evolution equations of: (i) joint decoding of irregular 
LDPC codes for a Slepian-Wolf problem with erasures, (ii) joint 
decoding of irregular LDPC codes on an erasure multiple-access 
channel, and (iii) general protograph codes on the BEC. This 
proves threshold saturation for these systems. 

Index Terms — convolutional LDPC codes, spatial coupling, 
threshold saturation, density evolution, potential functions 

I. Introduction 

Low-density parity-check (LDPC) convolutional codes, or 
spatially-coupled (SC) LDPC codes, were introduced in [1] 
and observed to have excellent belief-propagation (BP) thresh- 
olds in [2], [3], [4]. Recently, they have been observed to 
approach capacity for a variety of problems [4], [5], [6], [7], 
[8], [9], [10], [11]. 

The principle behind their excellent performance is de- 
scribed in [12], where it is shown analytically for the BEC 
that the BP threshold of a regular SC ensemble converges to 
the maximum-a-posteriori (MAP) threshold of the uncoupled 
ensemble. This phenomenon is now called threshold satura- 
tion. A similar observation was reported independently in [13] 
and stated as a conjecture. For binary-input memoryless sym- 
metric (BMS) channels, threshold saturation was empirically 
observed first [4], [5] and then shown analytically [11]. 

Threshold saturation now appears to be quite general and 
spatial-coupling has now been applied, with great success, to 
more general scenarios in information theory and coding [14], 
[15], [16], [17], [18], [19], [20], [21] 

Recently, the authors introduced a simple proof of threshold 
saturation for coupled scalar recursions where only a few 
details must be verified for each system [22]. The examples 
presented therein prove a number of threshold saturation 
conjectures made in the aforementioned papers (e.g., see [8], 
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[10]). The proof technique is based on potential functions 
for density-evolution (DE) recursions and was motivated by 
the ideas in [23]. Another approach to proving threshold 
saturation, based on a continuous approximation of spatial- 
coupling, has appeared in [21] and will appear in [24]. 

In this paper, the analysis is extended to prove that threshold 
saturation also occurs for certain coupled systems of vector 
recursions. In particular, if the single-system vector recursion 
can be generated by a scalar potential function, then one 
can also define a scalar potential function for the coupled 
system. Using this, one can show that threshold saturation 
occurs for a wider class of problems. For example, this settles 
conjectures made in [7], [25]. Along with the results in [26], 
this shows the universality of SC codes for a noisy Slepian- 
Wolf problem with erasures, when the channels are unknown 
at the transmitter. 

II. A Simple Proof of Threshold Saturation 

In this section, we provide a simple proof of threshold 
saturation via spatial-coupling for a broad class of vector 
recursions. The main tool is a potential theory for vector 
recursions that extends naturally to coupled systems. 

A. Notation 

The following notation is used throughout this paper. We let 
d G N be the dimension for the vector recursion, X = [0, l] d 
be the space on which the recursion is defined, and £ = [0, 1] 
be the parameter space of the recursive system. For conve- 
nience, we let X a = X \ {0} and S a = 8 \ {0}. Vectors 
are denoted in boldface lowercase (e.g. x,y), assumed to be 
row vectors, and partially ordered, x ^ y if Xi < yi for 
1 < i < d. Standard-weight typeface is used for scalar valued 
functions with a vector/matrix argument (e.g., F(x),F(X)) 
and boldface is used to denote a vector valued function 
with a vector argument (e.g., f(x) = [fi(x),--- ,fd{x)}). 
The gradient of a scalar function is defined by F'(x) = 
[dF(x)/dxi, ■ ■ ■ ,dF(x)/dxd] and the Jacobian of a vector 
function is defined by 

" dfi(x) _ d/i(x) ~ 
dx d 



/'(*) = 



df(x) 

dx 



dx d 



Matrices are denoted in boldface capital letters (e.g. X G 
X n ) and we use Xi = [X]i to denote the i-th row of X and 
Xi j = to denote the (i,j)-th element of X. Abusing 

notation, we also allow functions defined for vector arguments 



(e.g., f(x)) to act on matrices (e.g., f(X)) via the rule 
[f(X)]i = f(xi). The notation vec(X) denotes the transpose 
of the vector obtained by stacking the columns of X [27]. The 
Jacobian and Hessian of a matrix function are denoted by 

A dvec(f(X)) a dwcc(f(x)) 

f{X)= dvec(X) ^ f{x) = dx ' 

B. Single-System Potential 

First, we define potential functions for a class of vector 
recursions and discuss the associated thresholds. 

Definition 1: A vector admissible system (/,</), governed 
by the parameter e G £, is defined by the recursion 

x« + V = f(g{x<%e), (1) 

where / = [/i,--- , f d ] and g = [gi,--- ,g d ] are twice 
continuously differentiable strictly increasing in all arguments 
(w.r.t the partial order) vector valued functions defined com- 
ponentwise by fi : X x £ — > [0,1] and gi : X -> [0,1], 
1 < i < d. Furthermore, /(0;e) = f(x;0) = g(0) = and 
g'(x) is nonnegative and invertible for x G X\{0, 1}. 

Definition 2: The potential function U(x;e) of a vector 
admissible system (f,g) is 



U(x; e) 4 f \{z - f(g(z); e)) Dg'(z)} ■ dz 
Jo 

= g(x)Dx T - G(x) - F{g(x);e), 



(2) 



where F : X x £ — > M and G : X — > R are functionals 
that satisfy F(0) = 0, G(0) = 0, F'(x) = f(x)D, and 
G'(x) = g(x)D, for a positive diagonal matrix D G X d . 

Remark 1: The potential function is well-defined because 
the line integral is path independent. This property is easily 
verified by taking the derivative with respect to x. For DE 
equations, the potential function is closely related to the Bethe 
free energy (e.g., see [28, Part 2, pp. 62-65]). 

Definition 3: For x G X, and e G £. 

i) x is a fixed point (f.p.) if x = f(g(x);e). 

ii) a; is a stationary point (s.p.) if U'(x; e) = 0. 
Definition 4: The fixed-point set, T, its cc-support, Xf, and 

the epsilon set, e(x), are given by 

T^{(x,e) eX Q x£\x = f(g(x);e)} 
X f = {x G X | Be G £ s.t. (x,e) G J 7 }, 
e(x) = {ee£ | {x,e) G J"}. 

Lemma 1: For a vector admissible system, we have 

i) U(x; e) is strictly decreasing in e, for cc G X and e G 

ii) a; G X is a f.p. iff it is a s.p. of the potential. 

iii) For any x G Xf, e(x) has a single element. 

Proof: The potential function is a line integral of 
[z — f(g(z);e)) Dg'(z), and this expression is strictly de- 
creasing in e, for x G X and e E £. For the second property, 
we observe that the expression is zero iff z is a f.p. because 
Dg'(z) is always invertible for z G X\{0, 1}. For the third, 
let x G Aj\ By definition of Xf, e(x) has at least one element. 
Since f(x;s) is strictly increasing in e, for iel and e G £, 



a; = f(g(x)-e) can have at most one e-root. Therefore, e(x) 
is a singleton and we treat it as a function e : Xf — > £. ■ 

Definition 5: Let x G X, e G £, and x^ ^ = a;. Then, 
x°°(x;s) = lim^oo f(g(x^);s) (if it exists). 

Lemma 2: For all e E £, x°°(l;£) exists. 

Proof: Note that x^ = f(g(x^);e) < 1 = a;<°) as 1 
is the greatest element of X. It follows by induction on t that 
x (0) y X W y . . . y X W y . . . y q Hence, the sequence has 
a limit x^ = x°°(l;e). ■ 

Definition 6: The single-system threshold is defined to be 

s* ^ sup {e G f | x°°(l;s) = 0}, 

and is the e-threshold for convergence of the recursion to 0. 

Remark 2: The recursion (1) has no f.p.s in X iff e < e*. 
For DE recursions associated with BP decoding, the threshold 
e* is called the BP threshold. 

Definition 7: The basin of attraction for is defined by 

U x {e) = {x G X | x°°{x;s) = 0}. 

Note that U x (e) = X for £<£* but is a strict subset if £>£*. 
Definition 8: The potential threshold is defined by 



£* = sup{£ G £ | miria-g^- U(x;e) > 0}. 



(3) 



The energy gap is, for £>£*, AE(e) = M xeX \ UiB ^U(x;e) 
and £*<£<£* implies AE(e) > 0. Since U (x; e) is strictly 
decreasing in e, e* < e' for any e' that satisfies AE(e') = 0. 

Definition 9 ([29, Conj. 1]): Let the fixed-point potential, 
Q : Xf — > K, be defined by Q(a;) = U(x;e(x)). Then, we 
can define the Maxwell threshold £ Max to be 



£ Max A inf { £ g £ | ( aj) £ ) g j7 } = 0}. 



(4) 



Lemma 3: The potential threshold e* and the Maxwell 
threshold £ Max are equal. 

Proof: Any fixed-point x G A"/ is supported by a unique 
channel parameter e(x). Therefore, for x G Xf, we have 

U(x- e) = F(g(x); e(x)) - F(g(x);e) + Q(x) = Q(x) 



because £(a;) = e. Let x x satisfy (a; 



ax ^~Max 



G T 



and Q(a; Max ) = 0. Then, U(x Max ; £ Max ) = Q(a; Max ) = 0. 
From Def. 8, we know e* < £ Max . It can also be shown that 
A£(£ Max ) = and thus, £ Max < e*. This proves equality. ■ 

III. Coupled-System Potential 

We now extend our definition of potential functions to cou- 
pled systems of vector recursions. In particular, we consider 
a "spatial-coupling" of the single system recursion, (1), that 
gives rise to the recursion (5) and a closely related matrix 
recursion (6). For the matrix recursion of the coupled system, 
we define a potential function and show that, for e < e*, the 
only fixed point of the coupled system is the zero matrix. 

Definition 10 (cf. [12]): The basic spatially-coupled vector 
system is defined by placing 2L + 1 /-systems at positions in 
the set Cf = {—L, —L + 1,...,L} and coupling them with 
2L + w g-systems at positions in the set C g = {—L, —L + 
1, . . . ,L + (w — 1)}. For the coupled system, this leads to the 
recursion, for i G C g , given by 



±7^ 



w ^ 

k=0 



w — 1 

Es(* (£) 

3=0 



+j-k 



(5) 



where = e for « G = for i ^ Cf, x\ ^ = 1 for 

i <E £ g , and jcj^ = for i £ C g and all L 

Definition 11 (cf. [22]): The recursion in (5) can be rewrit- 
ten as a matrix recursion. Let X G x 2L+w have the decom- 



position X = [x~[ L , ■ ■ 



x I+w-i\ T - Then ' ( 5 ) is s iven b y 



A T f(Ag(X^);e), 



(6) 



where A is the following (2L + 1) x (2L + w) matrix. 

w 



A=- 

w 



1 1 




1 
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1 




1 
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••• 
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1 
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••• 
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... 1 



fcl 



2L + w 

Definition 12 (cf. [12]): Let i = L^J- The one-sided 
spatially-coupled vector system is a modification of (5) defined 
by fixing the values of positions outside £'j = {— L, . . . , io}. 

It forces the remaining values to x\^\ that is x^p — x\ e J for 
i < i < 2L + (w - 1) and all I. 

Lemma 4 (cf. [12, Lem. 14]): For both the basic and one- 
sided SC systems, the recursions are componentwise decreas- 
ing with iteration and converge to well-defined fixed points. 
The one-sided recursion in Def. 12 is also a componentwise 
upper bound on the basic SC recursion for i G C g and it 
converges to a non-decreasing fixed-point vector if L > io- 

Sketch of Proof: The proof follows from the monotonicity 
of f,g and a careful treatment of the boundary conditions. ■ 

Remark 3: This upper bound requires L > io = L^^J' 
to prevent influence of the right boundary. For the one-sided 
spatially coupled system with parameter e and a nontrivial f.p. 
X ^ 0, the io-th component satisfies Xi G X \ U x (e). 

Definition 13: The coupled-system potential for general 
matrix recursions is given by 

U(X-e) 4 Tr(g(X)DX T ) - G(X) - F(Ag(X);e), 

where G(X) = ^ G(x t ) and F(X; e) = F(x f , e). 
Remark 4: Using matrix derivatives, one can verify that 

[U'(X;s)]i - (xi - [A 1 'f(Ag(Xy,e)}i) Dg'{ Xl ). 

Definition 14: The down-shift operator S n : X n — > X n is 
defined by [S n X} 1 = and [S n X} i = x^i for 2 < i < n. In 
the sequel, the subscript of the down-shift operator is omitted, 
and it can be inferred from the context. 

Lemma 5: Let X G X n be a matrix with non-decreasing 
columns generated by averaging the rows of Z G X n over a 
sliding window of size w. Then, ||vec(S'X — ^)|| oc < — and 

llvectsx-x^HWIiHI^IL- 



Proof: For ||vec(5X - X)^, one has 



-E 



k=0 z i+k,j 



w Z^,k=0 * 



-l+fe,J 



< A. 



Since the columns of X are non-decreasing, the 1-norm sum 
telescopes and we get ||vec(S'X-X)|| 1 =||a; Il || 1 =||X|| oo . ■ 
Lemma 6: For any X G X 2L+W that satisfies [X] t = [X) io 
for i > i , we have U(SX;e) - U(X;e) < -U(x io ;e). 
Proof: First, we rewrite the potential as the summation 

^;£) = EST 1 [gixjDxJ-Gix^-FdAgiX^e)] . 

By virtue of the shift operator, [Ag(SX)]i = [SAg(X)]i. 
This observation implies that 

U(SX;e) - U(X;e) = -F ([Ag(SX)]_ L ; s)-U(x io ; e). 

Since F(x; e) > 0, this implies the stated result. ■ 
Lemma 7: For a fixed point of the one-sided SC system X, 

wec(U'(X; e)) ■ wec(SX - X) = 0. 

Proof: The first i + L + 1 rows of U'(X;e) are zero 
since X is a fixed point of the one-sided spatially-coupled 
system. Also, the rows i < i < 2L + (w — 1) of SX — X 
are zeros from the right boundary constraint. Hence, the inner 
product of these two terms is identically zero. ■ 
Lemma 8: The norm of the Hessian, U"(X;e), of the SC 
potential is bounded by a constant independent of L and w. 
Proof: By direct computation, we obtain 

< l|£>lloo(9m+9™ + 2(3™) 2 /™) =K f , g , 



s^p xeX \\g"(x) 



\\U"(X;e) 

where g' m = su Pa;GA . ||ff'(jc)| 
and f m = sup xex \\f'(x;e)\\^ 

Theorem 1: For a vector admissible system (f,g), with 
e < £*, w > (dK ftg )/(2AE(e)), and L > [^-\, the only 
fixed point of the spatially-coupled system (Def. 5) is 0. 

Proof: Let e < e* . Fix w > (dK f . g )/{2AE(e)) and L > 
L^J. Suppose X ^ is the unique fixed point (Lem. 4) of 
the one-sided recursion in Def. 12. Using Taylor's Theorem, 
the second-order expansion of U(SX; e) about X gives 

ivec(S'X - X) T C/"(C;e)vec(5 , X - X) 
= U(SX; e) - U(X; e) - wec(U'(X; e)) • vec(5'X - X) 
= U(SX; e) - U(X; e) ( Lem. 7 ) 
< -U(x io ;s) < -AE(e), ( Lem. 6 and Rem. 3, Def. 8 ) 

where C = X + t(SX - X) for some t G [0, 1]. Taking an 
absolute value and using the bounds in Lemmas 5 and 8 gives 

AE(e) < |ivec(S'X - x f U"(C;e)vec(SX - X)\ 

<i||vec(5X-X)|| 1 ||C/"(X; £ )|| oo ||vec(SX-X)|| oo 

< 



Therefore w < (dKf tg )/ (2AE(e)), a contradiction. Thus, the 
only fixed point for the one-sided spatially-coupled system 
is the trivial fixed point 0. The one-sided spatially-coupled 
system upper bounds the two-sided system, since L > L^^^J • 
Hence, the only fixed point of the two-sided system is 0. ■ 

IV. Applications 

In this section, we apply Theorem 1 to a few coding 
problems that have vector DE recursions. To save space, it 
relies on definitions and notation from [30], [31], [32]. 



A. Noisy Slepian-Wolf Problem with Erasures 

Two correlated discrete memoryless sources are encoded 
by two independent linear encoding functions, which are then 
transmitted through two independent erasure channels with 
erasure rates s\ and e 2 , respectively. We consider an erasure 
correlation model between the two sources. More specifically, 
let Z be a Bernoulli-p random variable such the two sources 
are the same Bernoulli- 1 random variable if Z = 1 and are 
i.i.d. Bernoulli-^ random variables if Z = 0. The decoder is 
assumed to have access to the side information Z. 

Assume that the i-th source sequence is mapped into an 
LDPC code with degree distribution (d.d.) [Xi,pi) and design 
rate 7 = 1 — L^(l)/i?-(l) using a punctured systematic 
encoder of rate 7/(1—7). The fraction of punctured systematic 
bits is 7 (see [7] for details). Let x\ ' (resp. x 2 ') be the average 
erasure rate of messages, from bit nodes to check nodes, 



corresponding to source 1 (resp. 2) and x^ = [x^\ x^p]. Let G(x) = J^ fe=1 L k {l) [x k + 



C : [0,1] -> [0, l] 2 , s ^ [e 1 (e),e 2 (e)\ with [ ei (0),e 2 (0)] = 0, 
be continuous and monotonically increasing. The DE recursion 
in [7] is easily generalized to asymmetric d.d.s and can be 
written in the form of (1) with 

ip(x; e) = (1 - 7)e + 7(1 - p + px), 

f(x;e) = [ip(L 2 (x 2 );e 1 (e))X 1 (x 1 ),'>p(L 1 (x-i);e 2 (e))X 2 (x 2 )], 
g(x) = [1 - pi(l - x-i), 1 - p 2 (l - x 2 )\. 

Using Def. 2 and D = diag (Li(l), L' 2 (l)), one finds that 

F(x;e) = tp(L 1 (x 1 ); e2(e))L 2 {x 2 ) 

+ ij){L 2 {x 2 ); si(e))Li(xi) - jpLi(xi)L 2 (x 2 ), 
Rk{l - x k ) - L 



--)■ 



For asymmetric d.d.s, one can also generalize both the trial 
entropy P(x), from [26, Lem. 4], and the mapping e(x) = 
[eW(a:),e [2] (aO], from [26, Sec. II- A]. This gives 

U(x;e) = (l- 7 )((e(x)-[e 1 (s),s 2 (s)])L(g(x)y-P(x)), 

where L(g(x)) = [Li(gi(xi)), £2(52(^2))]- Substituting e i-> 
e(x) implies that Q(x) = — (1 — -f)P(x). Therefore, we find 
that the Maxwell threshold from Def. 9 is equivalent to the 
standard Maxwell threshold [29, Conj. 1]. 

Corollary 1: Applying Theorem 1 shows that, if e < e Max 
and w > Kf, g /AE(e), then the SC Slepian-Wolf DE recur- 
sion must converge to the zero matrix. 

Remark 5: For special cases, one can use the methods 
in [26], [31], [32] to show that the Maxwell threshold defined 
above is an upper bound on the MAP threshold. These 
references also show that, for regular LDPC codes with fixed 
rate and increasing degrees, the upper bound approaches the 
information-theoretic limit. Therefore, SC regular LDPC codes 
are universal (e.g., see [7]) for this problem. 

B. Erasure Multiple-Access Channel 

We consider the two-user MAC channel with erasure noise 
(EMAC) from [25]. Let the inputs be X^,X^ e {±1} and 



the output be 

_ fxW + JfPl with probability 1 - e, 
1 ? with probability e 

Assume that the source sequences are encoded by LDPC 
codes with d.d.s (Ai,pi) and (A 2 ,/92). Let x^ (resp. x*ip) be 
the average erasure rate of messages from bit nodes to check 
nodes corresponding to user 1 (resp. 2) and x^ = [xf\ x^]. 
In [26], the DE recursion is written as (1), with 

ip(x; e) = e + (1 — e)x/2, 

f(x;s) = [ip(L 2 (x 2 );e)Xi(xi),ip(Li(xi);e)X 2 (x 2 )}, 
g{x) = [1 - pi(l - xi), 1 - p 2 (l - x 2 )\. 
Using Def. 2 and D = diag (L[(l), L' 2 (l)), one finds that 

F(x; e) = £[Li(a;i) + £2(2^2)] + (1 - e)L 1 (xi)L 2 (x 2 )/2, 
2 . . / R k {l-x k ) - 1 \ 

^ fc (l) >' 

Let the trial entropy, P(x), and the e(x) be defined by [26, 
Lem. 10]. Then, U(x;e) equals 

(e(x) - e)[L{g{x))l T - ^LMx^LMx^)} - P(x), 

and substituting e >-> e(x) implies that Q(x) = —P(x). 
Therefore, we find that the Maxwell threshold from Def. 9 
is equivalent to the standard Maxwell threshold [29, Conj. 1]. 

Corollary 2: Applying Theorem 1 shows that, if e < e Max 
and w > Kf t g/AE(s), then the SC DE recursion for the 
erasure MAC channel must converge to the zero matrix. 

Remark 6: For special cases, one can use the methods 
in [26], [31], [32] to show that the Maxwell threshold defined 
above is an upper bound on the MAP threshold. These 
references also show that, for regular LDPC codes with fixed 
rate and increasing degrees, the upper bound approaches the 
information-theoretic limit of the EMAC channel. 

C. General Protograph Codes on the BEC 

Consider the protograph ensemble [33] defined by an m x n 
protograph parity-check matrix H (e.g., H = [3 3] defines a 
(3,6)-regular code) and let [k] denote the set {1,2, ... ,k}. 
Let the dimension of the recursion, d, equal the number of 
non-zero entries in H and let the functions r : [d] — >• [m], 
c : [d] — > [n], and e : [d] — > {1, 2, . . .} map the index of each 
non-zero entry to its row, column, and value (i.e., e(k) = 
H r (k),c{k) for k G [d]). Let £j(s) be the erasure probability 
of the j-th bit node in the protograph as a function of the 
channel parameter e. Then, the bit- and check-node DE update 
functions f(x),g(x) are given by 

Mx; e)=e c{k) {e) ]J xf^ , 9k (x)=l- JJ {l-xtf™-*** 

i£[d]:c{i)=c{k) je[d]:r(j)=r( k ) 

where 8 it j is the Kronecker delta function. From this, we make 
an educated guess that the bit- and check-node potentials are 

F(x;e)=^e 3 (em t 



L i€[d]:c(i)=j 



G(x)=J2e(k)x k - 



fe=i 



E 

i=l 



n a 

je[d]:r(j)=i 



•U) 



Since each non-zero entry in H appears in only one row and 
one column, it is easy to verify that 

— F(x;e) = e(k)f k (x;e) and — G(x) = e(k)g k (x). 

This shows that one can choose D = diag (e(l), . . . , e(d)) 
and then apply Def. 2 to define a potential function for 
the protograph DE update. For the protograph ensemble, we 
conjecture that the fixed-point potential, Q(x), will also be a 
scalar multiple of the trial entropy defined by integration of 
the BP EXIT curve [30]. 

V. Conclusions 

Based on the work in [22], a new theorem is presented that 
provides a simple proof of threshold saturation for a broad 
class of vector recursions. The conditions of the theorem are 
verified for the density-evolution equations associated with: 
(i) irregular LDPC codes for a Slepian-Wolf problem with 
erasures, (ii) irregular LDPC codes on the erasure multiple- 
access channel, and (iii) general protograph codes on BEC. 
This provides the first proof of threshold saturation for these 
systems. Along with the results in [26], [31], [32], this also 
shows that SC codes are universal (e.g., see [7]) for the noisy 
Slepian-Wolf problem with erasures. 

Acknowledgment: The authors would like to thank Andrew 
J. Young for suggestions on and assistance with the final draft. 
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